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IMAGER LINKED WITH IMAGE PROCESSING STATION 



BACKGROUND 



1. 



Field 



This disclosure is related to the field of imaging. 



2. 



Background of the Invention 



When a digital video camera is aimed at a scene, it generates image data 
corresponding to the scene. The image data is then processed to produce an image of the 
scene. 

An effort is made to improve the quality of the eventual image. The effort is made by 
adjusting some of the operating parameters of the digital video camera. Some adjustments 
are made by the user, and some automatically by the camera. 

These operating parameters may include the pixel integration time, the gain, and the 
frame rate. The camera exposure parameters are the integration time and the gain. These 
parameters may be interrelated. For example, the integration time is currently determined, at 
least in part, by the luminance of the scene. If a scene starts becoming darker, the camera 
might correct for it by increasing the integration time. 

There are a number of criteria used to determine video image quality. One criterion is 
the video frame rate (the higher the frame rate, the higher the quality). Another criterion is 
noise. The lower the noise, the higher the image quality. The noise is quantified as a Signal 
to Noise Ratio (SNR) (the lower the noise, the higher the SNR). 

Often these two criteria are at odds. A higher frame rate requires a shorter integration 
time (and thus also a higher gain), which results in more noise (lower SNR). Achieving a 
higher SNR often requires a longer integration time, which in tum lowers the frame rate (and 
also requires lower gain). 

In some situations, an image processing station is tethered to, and made to operate 
with, the digital video camera. The camera generates and simultaneously transfers the image 
data to the image processing station over the tether, which can be a data cable. This is 
referred to as a linked digital video camera. 
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Additional parameters may be adjusted in the situation where a digital video camera is 
linked. Some of these additional parameters are typically controlled by special software 
residing in the image processing station, sometimes called the camera client. One of these 
parameters is the frame rate of the image data flowing through the communication link. In 
5 many instances, the frame rate is not controlled directly, but only indirectly. 

Sometimes the camera client requests the camera to transmit data at a specific frame 
rate. A digital video c£imera will attempt to deliver the frame rate requested by the camera 
client. Sometimes the requested frame rate is faster than can be accommodated by the 
bandwidth of the communication link. The requested frame rate will consequently not be 
10 provided. 

The invention will become more readily apparent from the following Detailed 
'2 Description, which proceeds with reference to the drawings, in which: 

' sr E 

\i I BRIEF DESCRIPTION OF THE DRAWINGS 

i y 

■B 15 Fig. 1 is a block diagram of an embodiment of a camera that has a communication 

• Q 

-J link with an image processing station via a data cable in accordance with the invention. 

!U Fig. 2 is a flowchart illustrating an embodiment of a method according to the 

;P invention. 

: E 

} o 

3 20 DETAILED DESCRIPTION 

Altemate embodiments of the invention may provide a system, software, and a 
method for controlling operating parameters of a digital video camera that is coupled with an 
image processing station via a communication link, although, of course, the invention is not 
thus limited in scope in this respect. 

25 Briefly the invention determines when the requested video frame rate is larger than 

the frame rate permitted by the limited bandwidth of the link. If that is so, the invention may 
set the integration time in such a way that the resulting frame rate is in accordance with a 
maximum frame rate permitted by the link. The invention may thus optimize the Signal to 
Noise Ratio (SNR), which also optimizes image quality. 

30 Referring to Fig. 1, an embodiment of a system according to the invention is now 

described. The system includes an article such as an image processing station 110. The 
image processing station 110 can comprise, for example, a special purpose device, or a 
general purpose device adapted for the special purpose. A general purpose device can 
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include a computer, such as a personal computer, in which case the adaptation can be by 
software. 

This particular embodiment can be used with an imager, such as digital video camera 
120. Here, camera 120 takes pictures of a certain frame size (FS). The frame size is a 
5 function of the image size and the image depth. The image size is the total numbers of pixels. 
For example, camera 120 can have an image size of 640 pixels wide by 480 pixels high. The 
image depth is a term of art for describing how many bits are used to describe the value of the 
image data of each pixel. A typical number might be 1 6 bits per pixel. In this case, the frame 
size (FS) therefore equals 640 x 480 x 16. Of course, any frame size may be employed in an 
10 alternative embodiment. 

Camera 120 is coupled to image processing station 1 10 by a communication link 130, 
which is also known as a data guide or simply a pipe. In the case of Fig. 1, the 
communication link 130 comprises a cable, coupled to station 1 10 at a port 140. Port 140 can 
comprise a Universal Serial Bus (USB) port. 
15 The communication link 130 can comprise any link, such as an infrared link, radio 

s| link, telephone line, Ethernet connection, or other data transfer link. In addition, the link 130 

can include a number of concatenated legs. For example, data might even pass through a 
network, such as the Intemet, before reaching station 110. 

The article of the invention also has a storage medium, such as a computer readable 
20 storage medium, and a computing device, such as a processor or controller, that can interact 
with the storage medium. The storage medium has stored thereon instructions of a program 
that can be executed by the computing device. In the embodiment of Fig. 1 , the image 
processing station 1 10 has a memory 150 and a Central Processing Unit (CPU) 160. 

The detailed description that follows is presented largely in terms of display images, 
25 algorithms, and symbolic representations of operations of data bits within the computer 

readable medium. These algorithmic descriptions and representations are the means used by 
those skilled in the data processing arts to most effectively convey the substance of their 
work to others skilled in the art. A person skilled in the art of programming can use this 
description to readily implement specific instructions for a program according to the present 
30 invention. 

Often, for the sake of convenience only, it is preferred to implement and describe a 
program as various interconnected distinct software modules or features, also known as 
software. This is not necessary, however, and there may be cases where modules are 
equivalently aggregated into a single program with unclear boundaries. 
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In any event, the software modules or features of this embodiment can be 
implemented by themselves, or in combination with others. Again, the combination can 
result in distinct software modules, or ones with blurred boundaries. 

An algorithm is here, and generally a self consistent sequence of steps leading to a 
desired result. These steps, also known as instructions, are those involving physical 
manipulations of physical quantities. Usually, though not necessarily, these quantities take 
the form of electrical or magnetic signals capable of being stored, transferred, combined, 
compared, and otherwise manipulated. When stored, they can be stored in any computer- 
readable medium. It is convenient at times, principally for reasons of common usage, to refer 
to these signals as bits, values, elements, symbols, characters, images, terms, numbers, or the 
like. It should be home in mind, however, that all of these and similar terms are associated 
with the appropriate physical quantities and are merely convenient labels applied to these 
quantities. 

In one embodiment, the method of the invention is implemented as machine 
operations optionally performed in conjunction with a human operator. Useftil machines for 
implementing the method include general purpose digital computers, the image processing 
station 110, the camera 120, or other similar devices. There should be borne in mind the 
distinction between the method of operating a computer and the method of computation itself. 
This embodiment relates to method steps for operating a computer and processing electrical 
or other physical signals to generate other desired physical signals. 

As described above, another aspect of the invention also relates to apparatus for 
performing these operations. This apparatus may be specially constructed for a specific 
purpose, or it may comprise a general purpose computer selectively activated or reconfigured 
by a computer program stored in the computer. The processes presented herein are not 
inherently related to any particular computer or other apparatus. In particular, various 
general purpose machines may be used with programs in accordance with the teachings 
herein, or it may prove more convenient to construct more specialized apparatus to perform 
the required method steps. The desired structure for a variety of these machines will appear 
from the description given below. 

The program of the invention can be stored in a computer-readable medium, such as a 
computer memory, a microprocessor, etc. But it should be clear to a person skilled in the art 
that the program of this embodiment need not reside in a single memory, or even a single 
machine. Various portions, modules or features of it can reside in separate memories, or even 
separate machines. The separate machines may be linked directly, or through a network, 
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such as a local access network (LAN), or a global network, such as what is presently known 
as Internet- 1 . Similarly, the users need not be collocated with each other, but each only with 
a machine that houses a portion of the program. 

An adaptable station 110 can run such a program for any camera to which it is linked. 
5 The program outputs at least one operating parameter for driving the camera 120. Any event 
may trigger running the program, such as requesting a new frame rate. The request can be 
generated extemally, or internally by the program. The request can be made any time. A 
new request can be looked for at any time, such as at video capture start, or when a reconnect 
is detected after a disconnect at a port 140 of station 110. 
10 In the embodiment of Fig. 1, the program can include two components. The first 

component is an application software 1 70 called the camera client. The second component is 
;g a camera driver 180. Other components, also known as layers, may be interposed between 

\il the apphcation software 170 and the camera dnver 180. 

ilj The program of the invention includes software for computing a bandwidth 

;Q 15 constrained frame rate (BCFR) from a frame size (FS) of the video camera, and a bandwidth 
(BW) of the link. Computation may be according to Equation (1) below. 

J BCFR = BW / [FS X CR] Equation (1) 

Q 20 where CR is the compression ratio. If there is no image compression, then CR may be set 
equal to one. 

The program additionally includes software for determining whether the computed 
BCFR is smaller than a requested rate of video frames from the camera, and if so, for 
determining an integration time of pixels of the camera from the computed BCFR. 
25 In one embodiment, the software for determining the integration time results in an 

integration time determined such that the camera outputs video frames at a rate 
commensurate with the computed bandwidth constrained frame rate. ,In other words, the 
integration time is computed such that the resulting video frame rate is the largest possible 
that the bandwidth of the link will permit. For example, if the BCFR is computed at 15 
30 frames per second, then the integration time becomes 1/15^^ sec. 

It will be appreciated that making the actual video frame rate commensurate with the 
BCFR is accomplished by maximizing the integration time. This minimizes noise, which 
optimizes image quality. 
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In some instances, where the target scene is very bright, the integration time may not 
be brought as high as the computed BCFR dictates. In any event, the SNR is still lov^. 

In one more embodiment, the software for computing the BCFR may use a 
compression ratio of the video camera. 
5 In a specific embodiment, the software for determining an integration time computes 

a numerical inverse of the computed bandwidth constrained frame rate. The inverse may be 
used for determining an integration time threshold (ITT). 

In an embodiment of the invention, the program fiirther includes software for 
determining a gain of the camera from the determined integration time. The gain thus 
10 determined compensates for the adjusted integration time. 

In some embodiments according to the invention, there may be predefined, associated 
values of integration time and gain, which may fiirther correspond to luminance. They may 

•O 

i H be associated explicitly, such as by being stored in a memory as a look up table with indices. 

ill 

iij Or they may be associated implicitly and indirectly, as values that result as pairs from sets of 

15 instructions that search through successive steps. 
SJ In all these cases, the invention may operate by temporarily constraining the choices 

of the camera, while allowing it to operate regularly otherwise. For example, the invention 
;^ may temporarily disable a portion of the table with integration times less than the determined 

=0 integration time threshold (ITT). Or it can reject results from instructions that render too low 

* S 20 integration times. 

Given the above, the instructions stored on the mediimi, when executed by a 
computing device, result in computing a bandwidth constrained frame rate (BCFR) from the 
frame size of the video camera 120 and the bandwidth of the link 130. The instructions 
additionally determine whether the computed BCFR is smaller than a requested rate of video 
25 frames from the camera, and if so, fiirther determine an integration time of pixels of the 
camera 120 from the computed BCFR. 

In an embodiment, executing the instructions may additionally result in the 
determined integration time to be such that the camera outputs video frames at a rate 
commensurate with the bandwidth constrained frame rate. 
30 Executing the instructions may also result in the bandwidth constrained frame rate to 

be computed also from a compression ratio of the video camera. 

In an embodiment, executing the instructions may result in the computing a numerical 
inverse of the computed bandwidth constrained frame rate. 
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Executing the instructions may further r.esult in determining a gain of the camera from 
the determined integration time. 

Embodiments of methods for computing operating parameters in a digital video 
camera in accordance with the invention include the following. In these embodiments, the 
5 camera is linked with an image processing station by a communication link, which has 
limited bandwidth. 

Referring now to Fig. 2, a representative method of the invention is described. 
References are made to components of this embodiment in accordance with the invention as 
they appear in Fig. 1 . In addition, it will be appreciated that Fig. 2 can also serve as a flow 
10 chart of a software according to the invention. 

According to box 210, a frame size FS of an imager such as the digital video camera 
•5 120 is set or determined. This can be by the camera client, either by inquiry or be a 

i n preprogrammed value. 

|S s According to box 220, a compression ratio CR of the digital video camera 120 is 

■y 15 determined. This is performed if a compression feature is provided. The program generally 

: rs 

knows in advance if the feature is provided, and if so, its value. Altemately, it can be 
determined by a query. 

p According to box 230, a bandwidth BW of the commimication link 130 is determined, 

.g If the port 140 includes a USB type connection, then the bandwidth is known. The 

20 bandwidth could change, e.g. if the link were through the internet. In the latter case the 
bandwidth may be queried, and the program can adjust the frame rate dynamically. 

According to box 240, a bandwidth constrained frame rate BCFR is computed from 
the determined bandwidth BW and frame size FS. Computation can be as taught in Equation 
(1). Although altemative approaches are also available, the computation can also include the 
25 compression ratio CR, if a compression feature is provided. If not, then CR is set equal to 1 
in Equation (1) in this embodiment. 

According to box 250, a requested frame rate is input. It can be requested by the 
camera client, at power up or other time. Or it can be input externally, at any time. 

According to a box 260, it is determined whether the requested frame rate is larger 
30 than the bandwidth constrained frame rate BCFR. If not, then the request is honored. If it is 
larger, however, then an integration time of pixels of the camera is determined from the 
computed bandwidth constrained frame rate. In some embodiments, the integration time is 
determined with a value that results in the camera outputting video frames at a rate 
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commensurate with the bandwidth constrained frame rate. In one of those embodiments, 
execution proceeds from box 260 to box 270. 

According to box 270, an integration time threshold ITT is determined. It can be 
computed as a numerical inverse of the bandwidth constrained frame rate BCFR, although the 
invention is not limited in scope in this respect. 

According to box 280, an integration time for the digital video camera 120 is set at a 
value at least as large as the determined integration time. It can be one of a set of preset 
acceptable values, or found by gradual adjustment. 

According to an optional box 290, an updated gain value is determined from the 
determined integration time. This can be performed in a manner similar to what was 
described above. The operations of boxes 280 and 290 can take place concurrently. 

The values can then be output to drive the camera 120. 

A person skilled in the art will be able to practice the present invention in view of the 
description present in this document, which is to be taken as a whole. Numerous details have 
been set forth in order to provide a more thorough understanding of the invention. In other 
instances, well-known features have not been described in detail in order not to obscure 
unnecessarily the invention. 

The specific embodiments thereof as disclosed and illustrated herein are not to be 
considered in a limiting sense. Indeed, it should be readily apparent to those skilled in the art 
in view of the present description that the invention can be modified in numerous ways. The 
inventor regards the subject matter of the invention to include all combinations and 
subcombinations of the various elements, features, fimctions and/or properties disclosed 
herein. 

The following claims define certain combinations and subcombinations, which are 
regarded as novel and non-obvious. Additional claims for other combinations and 
subcombinations of features, functions, elements and/or properties may be presented in this 
or a related document. 
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